Since its creation in 2011, Google Brain has been a key project in the development of machine learning and artificial intelligence (AI). This initiative, led by Andrew Ng, Jeff Dean, and Greg Corrado, has fueled progress in areas such as natural language processing, computer vision, and machine translation. Google Brain was born as a research team dedicated to exploring new ways to implement deep learning and neural networks in AI systems. As the project progressed, significant advances were made in the ability of machines to learn from large data sets without the need to program task-specific rules. In 2012, Google Brain made a breakthrough in image recognition, training a neural network on millions of YouTube images that, without prior knowledge, was able to identify cats with high accuracy.

The ability of computers to understand and generate language took a huge leap forward in 2017 when researchers at Google developed new natural -anguage AI models called Transformers. Some of the experts who built and trained those seminal models have since branched out on their own by founding the Toronto-based startup Cohere, which today announced a new $40 million Series A funding round. The technology that undergirds Cohere's natural-language processing models was originally developed by the Toronto-based Google Brain team. Two of that team's members, Aidan Gomez and Nick Frosst (along with a third cofounder, Ivan Zhang), started Cohere two years ago to further develop and commercialize the models, which are delivered to customers through an API. Cohere is backed by neural network pioneer and Turing Award winner Geoffrey Hinton, who led the Toronto Google Brain team, as well as some other big names in the AI world like Stanford computer science professor Fei-Fei Li. "Very large language models are now giving computers a much better understanding of human communication," Hinton said in a statement to Fast Company.

I recently started an AI-focused educational newsletter, that already has over 90,000 subscribers. TheSequence is a no-BS (meaning no hype, no news etc) ML-oriented newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Interpreting and understanding the behavior of deep neural networks remains one of the main challenges in the deep learning ecosystem. As humans, we regularly try to understand new subjects by comparing them to other knowledge areas we are familiar with.

Researchers at Google Brain have open-sourced the Switch Transformer, a natural-language processing (NLP) AI model. The model scales up to 1.6T parameters and improves training time up to 7x compared to the T5 NLP model, with comparable accuracy. The team described the model in a paper published on arXiv. The Switch Transformer uses a mixture-of-experts (MoE) paradigm to combine several Transformer attention blocks. Because only a subset of the model is used to process a given input, the number of model parameters can be increased while holding computational cost steady.

Trying to read this article is a complicated task from the neuroscientific standpoint. At this time you are probably bombarded with emails, news, notifications on our phone, the usual annoying coworker interrupting and other distractions that cause your brain to spin on many directions. In order to read this tiny article or perform many other cognitive tasks, you need to focus, you need attention. Attention is a cognitive skill that is pivotal to the formation of knowledge. However, the dynamics of attention have remained a mystery to neuroscientists for centuries and, just recently, that we have had major breakthroughs that help to explain how attention works.

Interpretability remains one of the biggest challenges of modern deep learning applications. The recent advancements in computation models and deep learning research have enabled the creation of highly sophisticated models that can include thousands of hidden layers and tens of millions of neurons. While its relatively simple to create incredibly advanced deep neural network models, its understanding how those models create and use knowledge remains a challenge. Recently, researchers from the Google Brain team published a paper proposing a new method called Concept Activation Vectors(CAVs) that takes a new angle to the interpretability of deep learning models. To understand the CAV technique, it is important to understand the nature of the interpretability challenge in deep learning models.

Last year in August Google released the Transformer, a novel neural network architecture based on a self-attention mechanism particularly well suited for language understanding. Before the Transformer, most neural network based approaches to machine translation relied on recurrent neural networks (RNNs) which operated sequentially using recurrence. In contrast to RNN-based approaches, the Transformer used no recurrence, instead it processed all words or symbols in the sequence and let each word attend the other word over multiple processing steps using a self-attention mechanism to incorporate context from words farther away. This approach led Transformer to train the recurrent models much faster and yield better translation results than RNNs. "However, on smaller and more structured language understanding tasks, or even simple algorithmic tasks such as copying a string (e.g. to transform an input of "abc" to "abcabc"), the Transformer does not perform very well.", says Stephan Gouws and Mostafa Dehghani from the Google Brain team. Hence this year the team has come up with Universal Transformers, an extension to standard Transformer which is computationally universal using a novel and efficient flavor of parallel-in-time recurrence.

Programmer Jeff Dean was one of Google's earliest employees, and is credited with helping to create some of the fundamental technologies that powered the tech giant's rise in the early 2000s. Now he's been put in charge of Google's future -- taking over as head of the company's artificial intelligence unit. The move is part of a reshuffle at Google, first reported by The Information and confirmed by CNBC, that's seems designed to push AI into more of the company's products. Previously, AI product development was overseen along with search by senior vice president of engineering, John Giannandrea . Now, this role is being split into two, with Dean taking over AI, and Ben Gomes leading the development of search.

The Google Brain Team looks back at a year of research ranging wrong their efforts to automate machine learning, understanding and generating speech and applying deep learning in computer systems and privacy and security.

Google's AI system assisted researchers in New Zealand in identifying calls of native birds -- Kakariki and Hihi -- using acoustic sensors after sifting through 15,000 hours of audio captured in and around Wellington. The Google Brain team, a core group focused on deep learning, used a trained Tensorflow model to label spectrograms and validate results to classify bird songs in real time. Tensorflow is an open source software for machine learning (ML) developed by the Google Brain team that was launched in 2015. Since then, Google has been running ML on different data sets -- from tracking seacows to diagnosing diabetic retinopathy and other health challenges. Linne Ha, director of Google Research and Machine Intelligence, shared updates on Google's ambitious Project Unison that's attempting to create text-tospeech (TTS) voices for lowresourced languages. There are 6,000 languages globally and 400 of them have over a million speakers.